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PREFACE 


This  study  has  been  a  fascinating,  challenging,  at  times  exhausting,  and  always  an 
exhilarating  experience.  Fascinating  because  it  opened  up  an  area  of  engineering  that  I 
formerly  viewed  as  "black  magic."  Challenging  because  my  weakest  areas,  prior  to  this 
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"fire  hose."  However,  it  was  always  exhilarating  to  learn,  understand,  and  even  ^ply  concq)ts 
that  I  formerly  viewed  as  wdl  beyond  my  capacity  to  conq)rebend. 

There  are  of  course  many,  many  people  to  whom  I  am  indebted  fcM*  suppcat  and 
encouragement  throughout  this  research.  Hrst  and  frxemost  I  want  to  thank  my  wife,  Kathy, 
for  putting  up  with  my  highly  volatile  and  unpredictable  schedule,  fw  listening  to  my 
engineering  banter  and  even  pretending  to  be  invested,  and  fm  reorganizing  my  prirmties 
when  they  really  needed  it  Next  ate  my  girls,  Amy,  Rachel,  and  Becka,  who  put  up  widi  a 
daddy  that  would  stare  at  the  computer  screen  and  say  "uh  huh"  or  "go  watch  TV"  whenever 
they  asked  any  questions,  and  who  would  occasionally  get  me  to  read  srmiething  other  than  a 
textbook  (although  textbooks  work  great  as  nighttime  stories,  puts  them  right  out).  I  want  to 
thank  my  thesis  advisor  Dr.  Peter  Maybeck  for  tte  prayos  (Lord  knows  I  iKeded  them), 
confidence  in  me,  guidance,  understanding,  slaps  on  the  back,  kicks  in  the  pants,  etc.,  etc. ... 
well  you  get  the  idea.  Mosdy,  I  want  to  thank  him  for  his  enthusiasm  for  tearhing,  without 
bis  animated  lectures  I  would  nevo*  have  undostood  the  foundational  material  for  this 
research,  and  Ms  violation  of  the  Principle  of  Conservation  of  Misery  (if  I  had  to  go  tiuough 
tMs  misery,  then,  by  God.  you  will  too)  that  seems  all  too  prevalent  in  engineeriiig  educatiort 
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Finally,  I  want  to  thank  the  my  fellow  students  that  suffered  through  the  controls  sequence 
with  me.  The  cohesiveness  and  mutual  assistance  that  we  built  during  our  18  month 
internment  dianged  this  experience  from  something  to  be  tolerated,  into  something  that  we  can 
look  back  on  (thank  goodness)  and  acmally  have  fond  memories. 
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ABSTRACT 

This  study  develops  and  investigates  tbe  perfonnance  of  a  Multiple  Modd  Adaptive 
Estimator  (MMAE)  to  detect  and  identify  control  surface  and  sensor  failures  on  the  LAMBDA 
flight  vehide  (a  DRV  devdoped  by  Wright  Latxx'atmies).  The  MMAE  uses  a  bank  of 
Kalman  filters  that  predia  the  airaaft  response  to  a  givmi  ii:q;>ut,  with  each  filter  modd  based 
on  a  differmit  failure  hypothesis,  and  then  forms  the  residual  difference  between  the  prediction 
and  sensor  measurements  for  each  filter.  Hie  MMAE  uses  these  residuals  to  determine  the 
probabilities  of  the  failures  that  are  modded  by  the  Kalman  filters.  Initially  the  MMAE 
identified  all  these  failures  within  4  seconds  of  onset  Various  performance  improvement 
techniques  were  researched  and  the  identification  time  was  reduced  to  less  than  2  seconds  after 
failure  onset.  This  improvement  was  mostly  due  to  an  increase  in  the  penalty  for  measurement 
differraices,  and  through  retiming  of  the  Kalman  filters.  The  MMAE  perftmnance  was  tested 
at  the  boundaries  of  the  LAMBDA  flight  envdqie,  with  good  perftxmance  found  at  points 
close  to  the  design  flight  conditioa  The  perftxmance  at  points  that  were  far  fix>m  the  design 
flight  condition  indicates  that  gain  scheduling  is  required  to  provide  adequate  perfwmance 
across  the  entire  envdope. 
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FAE.URE  IDENTIFICATION  USING 
MULTIPLE  MODEL  ADAPTIVE  ESTIMATION 
FOR  THE  LAMBDA  FUGHT  VEHICLE 


Chapter  I:  Introduction 


1.1  Chapter  Overview 

In  diis  cbapter  we  introduce  the  develc^ment  of  a  Multiple  Modd  Adaptive  Estimatcx’ 
(MMAE)  for  the  LAMBDA  flight  vehide.  In  Section  1.2  we  present  the  motivation  for 
devdoping  the  MMAE,  and  show  the  rdationship  of  diis  research  witih  contemporary  researdi 
efforts  in  this  area.  In  Sections  1.3  through  1.7,  we  define  die  details  of  the  research  problem 
by  first  defining  the  prodem  statement,  then  desoilnng  the  assumptions  made  to  accomplish 
this  research,  followed  by  the  specific  research  questions  diat  will  be  studied,  and  then  a 
description  of  the  so(^  and  limits  of  this  research.  Anally,  in  Section  1.8  we  present  an 
overview  of  the  structure  of  tins  thesis. 

1.2  Problem  Perspective 

Throughout  the  history  of  aircraft  devdqiment,  there  has  been  a  contiimous  push  for 
higher  performance  in  all  measures  of  aircraft  amiabilities  (^leed,  altitude,  maneuverability, 
accdoution  etc.).  To  obtain  dramatic  increases  in  performance,  designers  have  had  to  create 
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increasingly  complex  flight  control  systems.  For  example,  the  Wright  brother’s  major 
contribution  to  flight  was  the  design  of  an  aircraft  that  was  just  slightly  unstable  but 
controllable  by  a  human  pilot.  Since  the  instability  took  several  seconds  to  grow,  the  pilot 
could  easily  countermand  this  growing  instability,  thus  producing  stable  flight  [2:1-2].  Prior 
attempts  at  powered  flight  were  made  with  airaaft  that  were  inherently  stable,  which  proved  to 
be  unresponsive  and  impossible  for  a  human  pilot  to  fly.  Subsequent  years  saw  remarkable 
improvements  in  the  flight  control  area,  primarily  in  structural  and  mechanical  fields,  as  the 
fundamentals  of  flight  were  investigated  and  exploited. 

Eventually,  improvements  in  aircraft  performance  were  Umited  by  the  c^ability  of  a 
human  pilot  to  control  the  instabilities  that  were  necessary  to  produce  the  desired  inqrrovement 
in  performance.  At  first,  pilots  were  screened  and  improved  through  flight  training,  and  the 
best  pilots  were  assigned  the  highest  performance  aircraft,  usually  fighter  or  test  aircraft.  The 
mystique  of  these  elite  pilots  is  mostly  due  to  their  capability  to  adjust  thdr  reactions 
appropriately  to  the  aircraft  and  flight  conditions  that  they  encountered.  In  particular,  test 
pilots  were  legendary  for  their  ability  to  control  high  performance  aircraft  despite  difficult  and 
daiigerous  handling  qualities  or  various  faults  and  failures  that  they  encountered  during  flight 
testing.  The  demand  for  even  higher  performance  aircraft,  such  as  the  F-16,  that  have 
instabilities  well  beyond  the  capabilities  of  a  hum;i..  pilot  to  control  without  a  control 
augmentation  system,  has  produced  a  demand  for  even  more  capable  flight  control  systems. 

FOTtunately.  the  explosive  improvement  in  computer  technology  has  enabled  the 
developmrat  of  these  sophisticated  flight  control  systmns.  The  computer  c^abilities  of  the 
current  flight  control  systems  are  the  dreams  of  the  flight  control  designers  just  a  decade  ago. 
In  particular,  the  development  of  the  micrq)rocessm  has  brought  the  coneqn  of  parallel 
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processing  to  fruition.  This  architecture  allows  each  avionics  system,  and  ev«i  subsyston,  to 
accomplish  its  own  processing  without  relying  on  a  central  computer,  which  is  the  primary 
performance  limitation  in  non-parallel  avionics  suites.  While  these  improvements  in  the 
hardware  of  flight  control  systems  have  enabled  a  tremendous  increase  in  flight  control 
ci^abilities,  further  research  and  development  of  the  software  algorithms  that  employ  parallel 
architecture  is  required  to  realize  the  full  potential  of  parallel  processing. 

The  Multiple  Model  Adtq)tive  Estimation  (MMA^  algorithm  [1,  21,  22,  24, 47] 
naturally  exploits  the  capabilities  of  parallel  processing.  Its  structure  is  shown  in  Figure  1.  It 
is  composed  of  a  bank  of  Kalman  filters,  each  imbued  with  its  own  model,  and  is  by  nature  a 
parallel  processing  algorithm.  Each  elemental  Kalman  filter  uses  its  own  modd,  along  with  a 
^ven  iiq)ut  (  u  ),  to  devdop  an  estimation  of  the  currmt  aircraft  states  ( x, ),  ind^ndent  of 
the  other  filters.  The  filter  then  uses  this  estimate,  along  with  the  current  measuremm  of 
those  states  ( z ).  to  finm  the  residual  ( r, ),  which  is  the  difference  between  the  measurement 
and  the  filter’s  prediction  of  the  measurements  befme  they  arrive.  The  residuals  from  the 
filters  are  used  by  the  hyptnhesis  testing  algorithm  as  a  rdative  iidication  of  how  dose  each 
of  the  filter  modds  are  to  the  true  model  The  smaller  the  residual  the  doser  the  filter  modd 
matches  the  true  modd.  Hie  hypothesis  testing  algwithm  first  scales  tiie  residuals  to  account 
for  various  uncertainties  and  noises  in  the  measurements,  and  then  conqiutes  the  conditional 
probatnlity  for  each  of  the  hypotheses  modded  in  the  bank  of  Kalman  filters  ( p, ).  These 
probabilities  are  then  used  to  weight  the  individual  Kalman  filter  state  estimates  to  produce  a 
bloided  estimate  of  the  true  states  (  )•  which  can  tiien  be  used  as  the  (qitimal  estimate  of 

the  states  by  a  control  system.  When  used  for  failure  identification,  each  of  the  Kalimm  filters 
would  modd  a  diffetent  failure  condition  and  tiie  residuals  from  each  filter  would  indicate  how 
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close  that  filter’s  modd  is  to  the  actual  failure  condition.  By  monitoring  these  residuals,  the 
hypothesis  tester  can  d^ennine  the  current  failure  status  of  the  aircraft 

The  ciy)abilities  of  the  MMAE  have  been  qxxadically  investigated  since  the  early 
seventies  [1,  IS,  22,  23,  24,  36,  37],  with  a  significant  increase  in  research  since  the 
development  of  microprooessms  and  distributed  computatirm  [6, 7, 10,  21,  30,  31,  32,  33, 37, 
39,  40,  41,  42,  44,  4S,  46]. 

One  particular  ciq)alrility  that  has  produced  significam  research  effort  is  the  ability  of 
the  MMAE  to  identify  various  flight  control  failures  [28,  29,  33,  34, 35,  38, 41, 42].  Nhich  of 
this  effort  has  been  through  conqnita  modeling  and  simulation,  since  the  adaptation  and 
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complexity  of  the  MMAE  does  not  lend  itself  to  simple,  straightforward  analysis.  Also,  while 
various  implementations  have  been  studied,  none  have  been  implemented  and  tested  in  actual 
airaaft  systems. 

This  research  advances  the  investigation  of  Multiple  Model  Adaptive  Estimation  by 
developing  a  MMAE  that  q)erates  onboard  the  LAMBDA  flight  test  vdiide  as  a  flight  control 
failure  detector,  in  pr^aration  for  flight  testing.  The  devek^ment  and  flight  testing  of  a 
MMAE  for  the  LAMBDA  will  provide  realistic  and  valuable  information  on  the  capabilities  of 
the  MMAE  to  detect  failures  in  flight  control  system  actuators  and/or  sensors,  and  provide  the 
grotmdwork  ftx:  future  investigations  that  Ixoaden  the  applications  of  the  MMAE. 

1.3  Problem  Statement 

It  is  intended  to  devdtq)  a  MMAE  algtxithm  to  (tetect  a  single  failure  of  a  flight 
surface  or  flight  sensor,  fw  flight  testing  on  the  LAMBDA  flight  vehide.  The  LAMBDA  is 
an  unmanned  research  vehide  (URV)  devdoped  by  the  Flight  Control  Division  of  the  Flight 
Dynamics  Directorate.  Wright  Laboratory,  as  an  affordable,  flexible  research  vehide  for  testing 
and  dononstrating  flight  control  concqns,  devices,  and  systems  [43].  Further,  it  is  desired  to 
investigate  the  MMAE’s  ability,  through  compute  simulation,  to  identify  failures  in  both 
lateral  and  longitudinal-ditectional  axes  at  various  flight  conditions. 

The  Flight  Dynamics  Laboratory  plans  to  flight  test  the  MMAE  design  on  the 
LAMBDA  flight  vdiide,  investigating  its  ability  to  detect  an  aileron  actuator  failure  at  a  flight 
condition  that  is  normally  encountered  in  the  average  flight  profile  for  the  LAMBDA.  The 
flight  coixlition  of  the  LAMBDA  is  determined  by  five  parameters;  the  aircraft  wdght, 
forward  vdodQr,  dynamic  pressure,  center  of  gravity,  and  trim  angle.  The  average  flight 
condition  parameters  are  a  wdght  of  200  lbs,  a  speed  of  160  ft/sec,  a  dynamic  jnessure  of 
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30.43  Ib^,  a  center  of  gravity  located  at  46.8  inches  from  the  airaaft  nose,  and  a  trim  angle 
of  zero  degrees.  The  LAMBDA  usually  flies  at  low  altitude;  therefore  the  dynamic  pressure 
(  )  is  directly  related  to  forward  velocity  (  v )  since  the  air  density  ( p  )  is  essentially 

constant.  The  trim  angle  is  usually  very  small  for  a  wdl  trimmed  aircraft;  therefore  we 
assume  a  trim  angle  of  zao.  Since  the  altitude  and  trim  angle  are  considered  constant,  we  end 
up  with  a  three  dimensional  flight  envdope  fm  the  LAMBDA  as  shown  in  Figure  2,  which 
also  shows  the  design  flight  condition.  This  flight  condition  is  considered  average  because  the 
LAMBDA  is  usually  flown  dose  to  maximum  vdocity,  at  low  altimde,  with  the  variation  in 
weight  primarily  dq)endent  on  the  amount  of  fud  consumed.  Therdore  the  design  flight 
conditimi  is  a  median  value  of  the  ommal  operating  points. 


a 


<W 


(Do) 


Figure  2.  LAMBDA  Flight  Envdope 
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1.4  General  Assumptions 

The  assumptions  used  fw  this  research  are  based  on  those  of  previous  investigations 
into  the  use  of  Multiple  Model  Adaptive  Control  (MMAC)  to  provide  flight  ctMitrol  during 
various  failure  conditions  in  the  STOL  F-15  [28.  38,  29,  41]  and  MMAE  to  detect  and  identify 
various  failure  conditions  in  the  VISTA  F-16  [33.  34,  35,  42],  The  primary  task  for  this  thesis 
is  failure  detection  and  isolation  for  the  LAMBDA  flight  vehicle.  We  assume  that  once  the 
failure  is  properly  identified,  a  flight  controller  with  sufBcient  robustness  will  continue  to 
provide  stable  flight  control.  Theref(»e,  we  will  design  a  MMAE  to  accomplish  the  failure 
detection  and  identification  task,  ratho*  than  a  MMAC  with  its  associated  elemental  controller 
designs. 

We  further  assume  that  the  linearized  model  for  the  LAMBDA  is  sufficient  for  the 
developmoit  of  the  MMAE  algorithm.  Embedded  within  this  linearized  model  are  sevotil 
other  assuiiq>tions  that  weare  used  to  develop  the  model  [43],  namely  lateral  and  longitudinal 
axis  decoupling  and  first  wder  actuattM-  dynamics.  The  linearized  model  was  develq)ed  by 
estimating  flight  coeffidents  fi'om  geometric  data  and  then  adjusting  these  estimates  using 
good  englneeiing  judgment  and  flight  test  data.  For  a  fully  functional  aircraft,  the  cross  axis 
coiq)ling  terms  are  so  small  that  they  produce  a  negligible  airoaft  response.  Currently,  all  test 
flights  have  been  conducted  using  a  fully  functional  aircraft,  and  therefore,  the  flight  test  data 
does  not  contain  enough  information  to  estimate  the  cross  axis  terms.  The  assumption  of  first 
order  actuate  dynamics  compares  quite  well  with  tiie  flight  test  data. 

To  model  single  actuator  failures,  we  have  assumed  that  a  singia  flight  control  surface 
failure  will  produce  half  of  the  expected  re^nse  fix>m  dual  control  surfaces.  For  instance.,  a 
certain  etevator  input  might  produce  a  10  degree  pitch-up  response.  We  have  assumed 
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fw  this  same  input,  a  failure  of  only  the  right  elevator  acmator  would  produce  half  of  this 
response  (S  degree  pitch-up)  since  only  the  left  elevator  actuator  would  be  functional,  thereby 
decreasing  the  actuator  surface  area  by  one  half. 

As  mentioned  above,  we  have  assumed  that  the  ooss-axis  coupling  terms  are 
negligihle  even  for  single  surface  failures,  primarily  because  there  is  insufficient  data  to 
estimate  these  terms  [43].  Using  our  example,  a  right  elevator  failure  would  produce  a  small 
yawing  and  rolling  response,  which  we  assume  to  be  negligible.  An  exception  to  this 
assumption  is  the  cross-coupling  between  the  ailerons  and  the  yaw  axis  because  the  long  yaw 
moment  arm  of  the  ailerons  could  easily  produce  a  significant  yaw.  Fortunately,  ttere  is 
sufficient  data  to  estimate  the  aileron’s  effect  on  il^  yaw  rate  ( r  )  and  the  mdder’s  effect  on 
the  ndl  rate  ( p  ).  At  the  design  point,  the  contribution  of  the  aileron  to  a  yaw  rate  is  10.5% 
ami  the  contribution  of  the  rudder  to  a  roll  rate  is  only  0.3%.  This  supports  our  assunq)tion 
that  the  cross-axis  terms  are  small.  However,  Swift  found  noticeable  roU^itch  coupling 
dynamics  for  the  LAMBDA,  but  was  unable  to  estimate  the  cross  coupling  terms  [43:3.50].  A 
flight  test  with  single  actuator  failures  should  provide  the  much  needed  data  fm  estimating 
these  cross-axis  coupling  terms,  and  the  aircraft  model  can  then  be  corrected  fix'  further  tests. 

1.5  Specific  Research  Questions 

In  this  section  we  present  the  spedfic  questions  that  will  be  used  to  guide  this 
research.  Ttese  research  questions  were  grouped  into  three  areas,  decision  convergence, 
decision  switdiing,  and  unmodded  effects.  This  san»  structure  will  be  used  in  Chqxer  IV, 
Results,  and  Chapter  V,  Condusirxis  and  Recommendations.  Fbr  each  of  these  areas,  we 
presoit  the  specific  research  questums  (in  italics),  followed  by  our  rationale  for  investigating 
the  answers  to  these  questions. 
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1.5.1  Decision  Convergence 

Does  the  MMAE  probability-based  failure  detection  algorithm  converge  to  a  specific 
failure  hypothesis? 

Does  the  MMAE  probability-based  failure  detection  algorithm  converge  to  the  correct 
failure  hypothesis? 

The  answers  to  these  questions  will  characterize  the  failure  detection  performance  fw 
the  MMAE.  We  are  attempting  to  develop  an  algtmthm  diat  not  only  identifies  the  failure 
status  of  the  LAMBDA  flight  vehicle,  but  does  not  exhibit  fluctuations  in  its  identification. 
For  this  implementation  to  be  useful,  it  must  convage  to  the  correct  failure  status,  and 
maintain  that  correct  identificatioa 

1.5.2  Decision  Switching 

Are  the  decision  convergence  rates  fast  enough  to  prevent  lengthy  (or  even 
momentary)  incorrect  failure  hypotheses  declarations  by  MMAE  failure  detection 
algorithm? 

Are  the  decision  convergence  rates  dependent  on  the  flight  control  failure? 

Cm  the  MMAE  correctly  identify  the  failure  status  despite  m  initial  or 
momentary  incorrect  identification  of  the  failure  status? 

We  need  to  develop  an  algorithm  that  not  only  identifies  the  cwrect  failure 
status,  but  also  identifies  it  quickly  in  Oder  to  minimize  the  time  lag  until  the 
appropriate  flight  control  is  applied. 

We  also  will  lay  the  groundwork  fix'  fiiture  investigations  into  multiple  failure 
detection  and  identification  for  the  LAMBDA.  If  the  MMAE  can  correctly  identify 
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the  failure  status,  independent  of  the  initial  status,  then  it  has  the  potential  of  switching 
between  hierarchical  banks  of  parallel  Kalman  liters  to  provide  identification  of 
multiple  failures.  This  potential  was  investigated  in  previous  research  [29,  33,  34,  35, 
41,  42],  but  we  must  verify  this  property  for  this  specific  implementation  of  the 
MMAE. 

1.5.3  Unmodeled  Effects 

Are  there  any  misidentifications  of  failure  status  due  to  unmodeled  phenomena? 

Will  constant  coefficients  in  the  parallel  Kalman  filter  models  provide  sifficient 
performance  for  failure  ident^cation,  or  will  gain  scheduling  of  these  coefficierus  to  the 
currera  flight  condition  (speed,  altitude,  vehicle  weight,  etc.)  be  required? 

Previous  research  [40]  has  indicated  that  the  MMAE  is  capable  of  (q)eradng  over  a 
large  variation  of  model  parameters,  such  as  a  flight  envdope,  by  usiag  a  moving  parameter 
window  iQjproach  shown  in  Figure  3.  With  this  ^proach  a  few  Kalman  filter  models  are  used 
to  bracket  the  system’s  qia-ating  point  in  the  parameter  space.  These  Kalman  filter  models 
are  then  adjusted  as  the  system’s  operating  point  moves  in  die  parameter  space,  such  as  a 
change  in  the  aircrafl’s  flight  condition.  This  allows  the  MMAE  to  delect  failures  over  a 
much  largo*  pmtion  of  the  flight  envelope.  We  are  designing  our  MMAE  at  one  specific 
deagn  p<nnt  (the  median  flight  condition),  but  we  need  to  find  the  boundaries  over  die  flight 
envelope  where  the  MMAE  performance  deterioates  to  the  point  where  a  shift  in  the  design 
point  is  required  to  maintain  the  desired  failure  identification  performance. 
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Figure  3.  Moving  Parameter 


1.6  Scope  of  Research 

This  research  is  primarily  directed  at  inqjlanaitation  and  testing  of  the  MMAE 
algcHithm  for  the  LAMBDA  flight  vdiide.  Our  goal  is  to  develop  an  algorithm  that  can 
identify  various  single  failure  modes.  i.e,  failures  of  surface  actuators  or  sensors  of  the 
LAMBDA  flight  vehicle. 

There  are  certain  aspects  of  MMAE  that  we  are  not  investigating.  We  are  not 
implementing  a  MMAE  that  will  identify  multiple  failures.  Tboefore,  we  will  not  be 
developing  a  hierarchy  of  Kalman  Alter  designs  to  identify  diese  multiple  failures.  However, 
we  will  investigate  the  MMAE  perfocmance  under  various  initial  conditions  to  verify  that  it 
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correcdy  identifies  the  failure,  independent  of  the  initial  conditions,  thus  laying  a  foundation 
for  future  research  of  mulfiple  failure  detection  for  the  LAMBDA  flight  vehicle. 

1.7  Research  Limitations 

The  most  significant  limitation  of  this  research  is  the  adequacy  of  the  linearized  model 
for  the  LAMBDA.  This  model  was  developed  using  geometric  measurements,  mass  estimates, 
and  some  adjustments  to  the  parameters  to  match  (H-eviously  reoxded  flight  data  [43].  This 
development  assumed  that  the  axes  were  completely  decoupled,  that  the  actuator  dynamics 
were  first  order,  and  that  control  surfaces  worked  in  tandwn  to  produce  the  various  observed 
responses.  We  are  interested  in  detecting  and  identifying  a  single  actuator  failure,  so  we  must 
modify  the  model  to  predict  these  single  actuatm'  responses  to  the  givoi  inputs.  We  will  use 
this  linearized  model  to  build  our  various  Kalman  filter  modds  by  rqxesenting  actuattx' 
failures  as  a  zeroing  out  of  the  tqipropriate  column  of  the  input  matrix  (  ),  and  saisw 

faihires  are  represoited  by  zeroing  out  the  ^propriate  row  of  tl^  measurement  matrix  (  H  ). 
Other  failure  modes  are  not  modeled  in  this  thesis. 

To  detect  a  single  actuator  failure,  we  require  a  model  that  accounts  for  the  effect  of 
an  iiq>ut  to  each  of  the  individual  surfaces.  For  exanq)le,  our  model  needs  to  account  for  an 
iiqmt  to  the  right  aileron,  indq)endait  of  what  the  left  aileron  is  doing,  and  vice  versa.  To 
devdop  dus  split  actuator  model,  we  will  assume  that  the  response  ftom  a  single  actuator  is 
one  half  of  the  e]q)ected  reqwnse  of  the  case  of  bodi  actuators  being  used.  Note  that  a  singtit 
failed  actuator  will  cross  couple  its  response  into  the  other  axes,  but  we  have  not  inrJtided 
these  effects  in  the  linearized  aircraft  model  because  we  don’t  have  enough  data  available  to 
estimate  these  cross  coupling  terms.  This  area  definitdy  needs  to  be  researched,  and  as  more 
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data  becomes  available  on  the  effects  of  these  cross  coupling  terms,  corrections  to  the  modd 
should  be  incorporated. 

1.8  Thesis  Overview 

In  the  subsequent  ch^ters  we  devdop  this  research  in  more  detail  Chapter  2  gives  a 
brief  background  on  the  development  of  Multiple  Model  Adaptive  Estimation  and/or  Control 
(MMAE^MMAC)  and  discusses  some  of  the  pr(^)erties  found  in  previous  research.  Ch^ter  3 
presents  our  research  methodology  by  discussing  the  devdopment  of  the  MMAE  algmithm  for 
this  implementation,  the  software  tools  used  to  test  this  implementation,  and  a  description  of 
the  various  performance  measures  used  to  characterize  the  ctq)abilities  of  the  MMAE.  Chapter 
4  presents  tl»  findings  from  our  attempt  to  enhance  the  MMAE’s  performance  through  various 
techniques,  and  to  answer  the  specific  research  questions  posed  in  Section  l.S.  Ch^ter  5 
sununarizes  these  findings  and  condudes  with  our  recommmidations  fm  fiirthm*  smdy. 
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Chapter  n:  Background 


2.1  Chapter  Overview 

In  this  Cb^ter  we  present  the  background  of  multiple  model  adaptation.  We  start 
with  a  description  of  the  development  of  multiple  model  adaptation  estimation,  followed  by  the 
development  of  multiple  model  adq)tive  control.  We  dose  with  a  brief  catalog  of  applications 
of  multiple  modd  adaptive  estimation. 

2.2  Multiple  Model  Adaptation  Development 

2.2.1  Multiple  Model  Adaptive  Estimation  Development 

Hie  use  of  multiple  filters  in  a  parallel  structure  to  generate  ad^tive  estimation 
algorithms  was  first  devekqied  by  D.  T.  Magill  [23].  He  arranged  a  numbo’  of  Kalman  filters, 
each  wifii  different  time  invariant  plant  models,  and  used  the  residuals  from  these  filters  to 
form  an  appropriately  wdghted  sum  of  the  Kalman  filter  estimates,  as  shown  in  Hgure  4.  He 
showed  that  tiiis  adiqitive  estimation  algmitiim  produced  the  q[)timal  estimate  in  the  minimum 
mean  square  error  smise  for  a  Gauss-Maikov  process.  This  work  was  extended  to  handle 
discrete  systems  by  C.  B.  Chang  and  M.  Athans  [3, 4]. 

The  properties  of  tiiis  multiple  modd  adaptive  estimation  concqit  were  fiirther 
devdoped  by  Lainiotis  and  others.  They  investigated  the  properties  of  this  concqit  when 
using  a  discrete  number  of  models  to  rqnesent  a  continuous  domain  of  plam  models  [21,  22]. 
With  others,  he  showed  that  good  performance  was  attained  with  tightly  tuned  Kalman  filters, 
but  these  filters  produced  erratic  bduvior  if  tiie  filter  with  the  correct  modd  was  not  in  the 
filter  bank  [6,  13]. 
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2.2.2  Multiple  Model  Adaptive  Control  Development 

M.  Athans  and  others  extended  this  a(Uq>tive  estimation  concept  to  ad^ve  control 
developed  for  the  flight  comrol  system  of  NASA’s  F-8C  flight  test  aircraft  [1].  They  weighted 
the  optimal  control  signals  generated  by  a  bank  of  linear  system-Quadratic  cost  ftmetion- 
Oaussian  noise  distributitm  (LQO)  controllers,  each  with  an  embedded  filter  uang  a 

different  plant  model,  and  then  summed  these  weighted  control  signals  together,  as  shown  in 
Rgure  3.  Th^  found  that  tiiis  algorithm  provided  good  arntrei  at  flight  ctmditicMis  that  were 
dose  to  tiw  design  conditions  of  the  Kalman  Alter  models,  as  long  as  the  system  was 
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Figure  5.  Multiple  Model  Ad^tive  Control  Algcnithm 


appropriately  exdted,  using  a  test  signal  or  dither,  to  attain  good  failure  identificatioa  They 
also  found  that  this  algtxithm  was  sensitive  to  high  frequoicy  noise,  specifically  strong  wind 
gusts,  and  required  low  pass  filtering  to  attain  good  performance. 

Greene  and  Willsl^  [11]  examined  and  defii^  staUlity  tenons  whov  the  multiple 
model  adtqnive  control  (MMAQ  algorithm  yields  non-osdllatory  responses.  Tb^  found  that 
the  stability  of  the  MMAC  was  determined  by  die  relation  betwemi  the  growth  rate  of  the  most 
unstable  mode  of  a  Kalman  filler  with  a  mismatched  model,  when  compared  to  the  truth 
model,  and  the  rate  of  decay  of  the  slowest  stable  mode  of  a  Kalman  filter  with  a  nmiching 
modeL  They  presented  a  method  of  computing  the  borders  where  the  algorithm  is  neutrally 
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stable,  which  tten  defines  the  "domain  of  attraction,”  or  the  region  where  the  MMAC’s 
response  does  not  oscillate. 

2.3  Multiple  Model  Adaptation  Applications 
2.3.1  Failure  Detection 

A.  S.  Willsky  [47]  surveyed  a  number  of  failure  detection  methods,  using  a 
performance  index  as  an  empirical  measurement  of  the  capabilities  of  the  various  methods. 
This  perfixmance  index  included  the  types  of  failure  modes  that  can  be  detected, 
implementation  conq>lexity,  various  performance  measures  (false  alarms,  d^ection  delays, 
rq)eatat^ty),  and  robustness  in  the  presence  of  modeling  errors.  He  found  that  the  nuiltiple 
model  adaptive  controller  will  yield  the  best  performance  over  the  widest  dass  of  failures. 

Longitudituil  contrd  of  the  Short  Take-Off  and  Landing  (STOL)  F-IS  using  Nhiltiple 
Model  Adaptive  Control  (MMAQ  was  investigated  by  Ppgoda  [28,  38]  and  Stevens  [29,  41]. 
Pogoda  devekqted  an  algorithm  fix  the  landing  phase  of  the  flight  regime,  which  would 
reconfigure  die  flight  control  in  die  presence  of  a  single  failed  control  surface  or  sensor.  He 
designed  Kalman  filter  models  fix  a  fully  fimcdonal  aircraft,  a  failed  stabilator,  a  failed 
"pseudo-surface,"  and  a  failed  pitch  rate  senstx.  The  "pseudo-surface"  was  a  combination  of 
the  canards,  ailerons,  and  fUyis,  to  allow  a  reducdon  in  the  number  of  aircraft  states.  Stevens 
mctended  this  research  further  by  including  failed  reverser  vanes,  a  failed  velocity  sensor,  and  a 
failed  flight  path  sensor.  He  first  investigated  "soft”  or  partial  failures  of  diese  soisors 
surfaces,  with  the  soft  failures  modeled  as  either  partial  power  to  a  flight  surface,  an  increase 
in  the  sensor  noise,  or  an  increase  in  the  senstx  bias.  He  then  investigated  the  performance  of 
a  hierarchical  structure  of  mult^le  model  ad^ve  controllers  to  detect  the  presence  of  double 


17 


failures,  such  as  a  stabilator  surface  failure  followed  by  a  flight  path  angle  s^isot  failure.  This 
hierarchical  structure  of  controllers  started  with  a  bank  of  controllers,  with  different  single 
failure  models,  that  would  detect  the  first  failure  and  thmi  switdi  to  another  bank  of  controllms 
that  had  models  that  assumed  both  the  detected  failure  and  a  second  failure.  Each  of  the 
banks  also  had  a  controller  with  a  failure  model  that  assumed  that  the  detected  failure  had  not 
occurred  after  all,  which  allowed  the  structure  to  correct  any  misidentifications.  Pogoda  and 
Stevens  both  found  that  the  MMAC  structure  was  able  to  identify  the  failures  properly  and 
reconfigure  the  control  law  to  maintain  stable  flight  control.  Stevens  found  that  the  MMAC 
would  blend  the  appropriate  controller  commands  in  the  presence  of  soft  failures,  and  that  the 
hierarchical  structure  would  properly  detect  multiple  failures. 

In  a  similar  manna,  Stratton  [42]  and  Menke  [33, 34, 35]  have  developed  a  MMAE 
algorithm  for  both  the  longimdinal  and  lateral  axes  of  the  VISTA  F-16  aircraft.  The  emphasis 
of  their  design  was  to  develop  a  failure  detector  that  would  work  with  an  existing  flight 
controller.  Therefore,  they  developed  a  MMAE  that  produced  the  best  estimate  of  the  failure 
status  of  the  aircraft  and  the  aircraft  states,  which  was  then  used  by  the  aircraft’s  ctmtroller  to 
provide  iQ)propriate  flight  control  in  the  presence  of  the  failure.  Stratton  used  a  flight 
condition  of  Mach  0.8  and  altitude  10,000  feet,  while  Menke  designed  for  a  flight  condition  of 
Mach  0.4  and  altitude  20,000  feet,  the  latter  invoiving  low  dynamic  pressures  and  thus 
presenting  a  more  difficult  failure  detection  problem.  They  investigated  both  single  and 
multiple,  hard  and  soft,  actuatcM’  and  sensa  failures.  Included  in  thdr  study  was  the  effect  of 
a  test  signal  or  commanded  ditlmr  to  aid  in  identifying  the  failure  during  benign  straight  and 
level  flight  conditions.  Several  dither  signals  were  tested,  including  sine  waves,  square  waves, 
triangular  waves,  and  pulse  trains,  at  levels  that  were  deemed  either  subliminal  (up  to  ±  0.1  g’s 
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in  the  longitudinal  axis  and  ±  0.2  g’s  in  the  lateral  axis)  or  non-subliminal  (reasonable  physical 
acceleration  limits  at  the  pilot’s  station).  They  fcmnd  that  the  MMAE  identified  these  failures 
correctly  as  long  as  an  appropriate  dither  signal  was  present  that  excited  all  failure  modes  in 
both  axes. 

2.3.2  Other  Applications 

Multiple  model  adaptive  algorithms  have  beoi  successfully  devdoped  for  a  number  of 
other  q)plications.  It  was  investigated  for  the  detection  and  tracking  of  maneuvmng  targm 
[5,  9,  10,  24,  30,  36,  37,  44,  45,  46],  flexible  space  structure  control  [8,  16,  17,  19,  20, 40], 
multiple  hypotheses  testing  [1,  31],  and  to  prevent  the  initial  divergence  of  extended  Kalman 
filters  due  to  large  initial  uncertainties  [IS,  32].  It  has  been  smdied  for  use  in  diverse 
implications  such  as  instrummit  failure  detection  in  a  pressurized  water  fusion  reactor  [7], 
autonomous  monitoring  of  cardiac  patients  [12],  adaptive  signal  processing  of  seismic  data 
[39],  and  detection  of  inddeaots  on  freeways  [48]. 
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Chapter  ni:  Methodology 


3.1  Chapter  Overview 

In  this  cb^ter  we  develop  the  MMAE  al^nithm  and  the  software  simulator  that  we 
used  to  test  the  MMAE.  We  start  with  an  overview  of  the  MMAE  structure  and  then  develop 
the  Kalman  filter  and  hypothesis  testing  algorithms.  We  then  describe  the  computation  of  the 
steady  state  Kalman  filter  gains.  Then  we  devdop  a  truth  modd  of  the  LAMBDA  that  is  used 
to  provide  the  MMAE  with  a  simulation  of  noise-corrupted  measurements  and  commanded 
inputs  during  performance  testing.  Lastly,  we  describe  the  various  ouqmts  that  were  used  as 
diagnostics  dining  the  software  devdt^mem  and  we  present  the  finmat  of  the  results  of  the 
performance  testing  that  will  be  used  in  Chapter  IV. 

3.2  Multiple  Model  Adaptive  Estimation  Algorithm 
3.2.1  Overview 

A  MMAE  condsts  of  a  bank  of  paraUd  Kalman  filters,  eadi  with  a  different  imernal 
modd,  and  a  hypothesis  conditional  probability  conqnitation  as  shown  in  Figure  6.  The 
Kalman  filters  are  provided  a  measurement  vectm  (  z )  and  die  itqput  vectm  (  v  ),  and  produce 
a  state  estimate  ( x, )  and  a  reddual  ( r, ).  Eadi  Kalman  filter  has  a  different  failure  modd 
that  it  uses  to  form  the  state  estimate  and  the  residual,  so  the  sizes  of  the  residuals  give  a 
rdadve  indication  of  how  wrong  each  of  these  models  are.  The  residuals  are  used  by  the 
hypothesis  testing  algorithm  to  asdgn  relative  probalrilities  ( p, )  to  each  of  die  hypotheses  that 
were  used  to  form  the  Kalman  filter  modds.  The  individual  probabilides  indicate  how  correct 
eadi  of  the  Kalman  filter  modds  are,  and  can  be  used  to  wdght  the  individual  srntp.  asHnmtwg 
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iy>propriatdy,  to  fonn  a  probability-weighted  avoage. 

An  example  would  demonstrate  the  workings  of  the  MMAE  more  dearly.  Let  us 
assign  Kalman  filter  1  the  fully  functional  aircraft  model,  and  Kalman  filter  2  the  left  devator 
failure.  Hiese  filters  would  use  the  commanded  iiqmt  and  compute  an  estimate  of  what  the 
state  measuronents  should  be.  Induded  in  this  computation  is  a  modd  of  the  noise  in  thiwm 
measurements,  which  the  filters  use  to  calculate  an  expected  value  and  standard  deviation  of 
the  measuremoit.  The  actual  measurement  is  dien  used  to  form  the  residual,  which  is  the 
difference  between  the  actual  measurement  and  what  the  filter  erq)ected  for  the  measuranent 
If  the  aircraft  has  no  failures,  then  die  residual  from  fiita*  1  would  be  much  smaller,  relative  to 
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its  own  internally  computed  standard  deviation,  than  the  residual  in  filter  2.  The  hypothesis 
testing  algorithm  would  take  all  of  the  residuals  and  assign  the  highest  probability  to  filter  1 
since  it  has  the  smallest  residual.  Now,  let  us  assume  that  a  left  elevator  failure  occurs,  thus 
the  residual  in  filter  2  would  become  quite  small  and  the  residual  in  filter  1  would  grow.  The 
hypothesis  testing  algorithm  would  then  assign  less  probability  to  filtm*  1  and  mme  to  filter  2. 
The  specific  workings  of  these  blocks  are  explaiited  in  the  following  sections. 

3.2.2  Kalman  Filters 

In  this  section  we  assume  that  the  reader  is  familiar  with  Kalman  filtering  and  present 
the  specific  Kalman  filter  equations  used  in  the  MMAE.  Maybeck  [24]  presents  an  excellent 
development  of  Kalman  filtering  theory,  if  further  Kalman  filtering  background  is  needed. 

We  are  implementing  this  algorithm  on  a  distal  conq)uter,  which  requires  a  discrete- 
time  system  modd,  and  we  also  need  to  minimize  the  conq)utational  loading.  Previous 
research  [43]  devdoped  a  continuous  time  system  modd  for  the  LAMBDA,  which  we  will 
convert  to  a  disa^time  equivalent  modd.  To  minimize  coiiq)utational  loading,  we  assmne 
that  the  system  is  time  invariant,  giving  us  constant,  precomputable  coeffidents  fix'  the  system 
modd  ( 4>,  H ),  and  we  use  a  linear,  steady  state  Kalman  filter,  which  gives  us 

constant  Kalman  filto*  gains  ( K ). 

In  Section  3.3.1  we  will  devdt^  a  discrete  time  equivaloit  syston  modd  of  the  form 

*(r,)  -  ®  *  Gg 

zit,)  -  + 

where  jc  is  the  system  state  vectm 

<t>  is  tlte  state  transition  matrix,  tiie  discrete  equivalent  of  the  system  dynamics  matrix 
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is  the  disCTete  equivalent  of  the  system  control  input  matrix 
If  is  the  system  iiqnit  vectm^ 

is  the  discrete  equivalent  noise  input  matrix 

is  an  additive  white  dynamics  noise  uq>ut  with  zero  mean  and 


h  =  *j 

0  t,*tj 


(2) 


z  is  the  measurement  vector 
H  is  the  system  ouq)ut  matrix 

V  is  an  additive  white  measuremoit  muse  iiq>ut,  indq)endeot  of  with  zero  mean  and 


Ritf)  t,  *  tj 

0  t^^tj 


(3) 


Using  this  model  we  get  the  Kalman  filter  state  estimate  pnqxagation  equation: 

i(r,-)  =  +  (4) 

where  is  the  time  just  before  the  rth  time  sample,  and 
t,.  *  is  the  time  just  after  the  i  -  1  time  sample, 
and  die  Kalman  filter  update  equation; 

i«r)  •  i(V)  ^  -  Biitp]  (5) 
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This  model  also  gives  us  the  Kalman  filto*  residual: 


r(r,)  (6) 

whidh  the  hypothesis  testing  algorithm  uses  as  a  rdative  measure  of  how  mudi  the  Kalman 
filter  model  differs  from  the  true  model.  The  residual  is  the  difference  between  the  true 
measiirementiis  ( z ),  and  the  Kalman  filter  estimates  of  those  measurements  befcue  they  are 
taken  ( Hx{ti) ).  which  are  based  on  its  model.  Thus,  if  the  Kalman  filter  model  is  correct,  the 
residual  will  be  dose  to  zero,  otherwise  the  residual  will  be  larger  than  antidpated  when 
compared  to  the  filter-computed  residual  covariance  matrix,  whidh  is  shown  in  (7)  bdow. 

3.2.3  Hypothesis  Testing 

Maybeck  shows  [24:228-229]  that  the  Kalman  filter  residual  is  a  white  Gaussian 
sequence  of  mean  zero  and  covariance 

(7) 

For  our  iq)plication,  which  assumes  a  time  invariam  system,  this  covariance  is  constant  and 
precomputable.  Fbr  the  k-th  elemental  Kalman  filter  the  covariance  is 

A*  =  +  (8) 

Since  we  know  that  the  residual  is  a  Gaussian  vector  with  zoo  mean  and  covariance  A^,  we 
substitute  these  values  into  the  known  expression  fix’  a  Gaussian  conditional  density  function. 
Therefore  we  get  that  the  conditional  density  fimction  of  the  measurement  ( z )  at  r, ,  given  the 
Kalman  filter  modd  ( )  and  the  measurement  history  (  Z(r,.  i)  =  [  z^(r, ) :  -  :  z^(r,.  i)] ),  is 
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~  Pexp^'^ 

p  = - 1 - 


(9) 


We  desire  tbe  conditional  probabilities  of  the  various  hypotheses,  p^ti ),  which  we  can 
compute  using  (9)  and  the  previous  conditional  probabilities  p^it^ . , ): 


Pk(h^  = 


fsit,) 

__ 

J-t 


(10) 


Stevens  [29, 41]  found  that  certain  poftvmance  problems  could  be  reduced  by 
modifying  (9).  He  altered  the  conditional  density  function  in  (9)  by  removing  titt  p  term, 
which  was  used  to  make  the  area  under  the  density  function  equal  to  1.  If  all  the  esqwnential 
terms,  { *}  in  (9),  were  {q)proximately  the  same  size  f(»-  all  donoital  filtm,  (9)  and  (10) 
would  put  the  highest  probability  on  the  elonental  filters  with  the  smallest  |At|  value.  This 
is  an  iniq)propriate  wdghting  since  the  size  of  |At  j  has  Mriiing  to  do  with  the  correctness  of 
the  hypothesis  in  matching  tbe  current  real-wold  failure  status.  Since  senso  failures  exhibit 
themsdves  as  a  row  of  H  gdng  to  zero,  filters  based  on  tbe  hypothesis  of  a  failed  sensor  tend 
to  have  smaller  |Ajt|  values,  and  thus  the  MMAE  will  be  prone  to  &lse  alarms  on  sensor 
failures.  The  al^xithm  functions  properly  with  tbe  P  term  removed  because  tiie 
in  (10)  is  the  sum  of  all  possiUe  numerators,  so  tbe  probabilities  ( )  will  still  sum  to  1  even 
if  the  area  under  each  of  tiie  modified  "densities,"  the  density  function  in  (9)  with  p  removed, 
is  no  longer  unity. 
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Previous  research  [3.  28,  29,  38, 41]  also  found  that  a  lower  bound  needed  to  be 
placed  on  the  probabilities.  Hie  purpose  of  the  MMAE  is  to  make  quick  and  accurate  failure 
identificatioiis,  and  it  was  found  that  if  some  of  the  probabilities  were  allowed  to  get  too  small, 
it  took  a  long  time  for  the  probability  in  the  correct  filter  to  build  iq>  when  a  change  in  the 
failure  status  occurred.  This  was  due  to  the  fact  that  the  previous  conditional  probability, 

Pk(h .  i)>  so  small  for  the  new  correct  filter  model  and  so  large  for  the  old  correct  model, 
that  (10)  had  to  be  iterated  several  times  before  the  values  would  change  significantly.  Menke 
and  Stratton  [33,  34, 35, 42]  found  that  0.001  was  a  good  lower  bound  on  tl»  conditional 
probabilities.  This  is  tl»  implementation  of  the  hypothesis  testing  algorithm  used  fca:  this 
application  of  the  MMAE. 

3.3  Multiple  Model  Adaptive  Estimator  Simulation 

Now  that  we  have  a  MMAE  algorithm,  we  need  to  develq)  a  simulatioii  diat  will 
characterize  the  performance  of  the  MMAE  for  the  LAMBDA  flight  vdude.  First,  we  will 
determine  the  precorrqnitable  coeffidoits  fm*  the  system  modd  ( O,  B^,  G,  H ),  and  the 
constant  Kalman  filter  gains  ( K ).  Then  we  will  develop  the  truth  modd  that  will  be  used  to 
genm^  the  measurements  of  the  aircraft  states  for  a  givmi  irq)ut  Lastly,  we  outline  the 
variables  that  were  used  as  diagnostic  tools  and  measures  of  perftnmance  for  the  MMAE. 

3.3.1  Steady  State  Kalman  Filter  Coefficients  Computation 

The  MMAE  algwithm  requires  several  pteoonq)utable  coeffidents  for  each  Kalman 
filter  modd.  To  obtain  these  coeffidents,  we  start  with  a  contirmous-time  modd  of  die  fixm 
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X  =  Px  +  Bu  +  Gw 


(11) 


This  contiauous-time  modd  for  tbe  LAMBDA  was  developed  by  Swift  [43]. 

We  then  fiiul  the  discrete-time  equivalent  system  model  shown  in  (1)  and  d^ne  the 
following  variables  from  these  equivalent  system  models: 

H,  the  measurement  matrix  in  (1)  is  defii^  by  the  measurements  of  the  Kalman  filter 
states  that  are  available, 

G,  the  white  noise  iiq)ut  matrix  in  (1 1)  is  defined  by  the  states  that  are  affected  by  the 
white  noise  iiq)ut  w, 

G^  the  discrete  time  equivalent  matrix  to  G  shown  in  (1).  whidi  is  defined  to  be  1  for 
the  discrete  time  equivalent  model, 

Q,  the  strength  of  the  additive  white  noise  ( w  )  in  (11),  which  is  defined  in  the  trudi 

model  development  (Section  3.3.2)  and  modified  if  the  Kdman  filters  need 
tuning  to  attain  good  poformance, 

R,  the  measurement  noise  covariance,  defined  by  estimates  of  the  noise  in  tte  various 

measurements  (i.e.,  soisor  precision)  ai^  modified  if  the  Kalman  filters  need 
tuning  to  attain  good  pmfmmance. 

The  estimates  of  B  were  enqniically  detomined  using  the  estimated  standard  deviation  of 
aircraft  measurement  data  taken  when  the  aircraft  was  quiescent  (sitting  on  the  ground  with  no 
movement). 

We  used  the  contiiiuous-time  state  modd  (1 1)  to  conqmte  the  state  transitimi  matrix 
( 4> ).  We  start  with  the  givmi  flight  condition,  conqmte  the  state  space  modd  of  the 
continuous-time  system,  zero  out  the  appropriate  column  or  row  to  simulate  the  desired  failure 
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model,  and  tben  use  a  MATRD^  intrinsic  function  (SPLIT)  to  find  the  disaete-time  equivalent 
<I>  fM*  this  system.  This  method  was  verified  by  propagating  x  for  sevaal  sample  times  using 
tlm  equation 

xit^)  =  fPxix)dx  +jc(to)  (12) 

and  comparing  its  value  against  the  discrete-tinK  equivalent  equation 

*(t,)  -  (13) 

With  0  evaluated  for  one  sanq)le  pmod  ( dKt,  ).  we  can  find  second  order 
approximations  to  and  Qg  where 

B^=  f  B^(t^,^-x)Bdx  »  ^(•B  +  B)Af 

t, 

(14) 

<?/-  /♦(«,.! 

»  I(*G<?C*'#*'  +  G<?G*’)Af 
2 

To  find  the  steady  state  icaimjin  filter  gains,  we  jnopagated  and  vpdated  the  filter 
covariance  using 

F(t-)  .  •P(fVi)*'  +  C,<?,G/  (15) 

for  the  inopagation  and 
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«(f,)  =Pitf)H^[HPitpH^^R]^ 
PiO  =  P(t,-) -Kit^)HP(tf) 


(16) 


for  tbe  update,  until  tbe  value  of  K  reached  steady  state.  This  gave  us  K„  and  P„',  whidi  we 
used  to  confute  and  its  inverse.  This  gave  us  all  the  ptecomputable  coefficients  fen'  the 
various  Kalman  filter  models,  which  were  then  stored  in  data  files  for  use  by  the  MMAE. 

3.3.2  Truth  Model  Development 

The  truth  model  was  used  to  generate  the  simulation  of  the  measurements  that  the 
LAMBDA  aircraft  would  normally  provide  the  MMAE.  The  truth  model  also  started  with  tbe 
continuous  time  system  model  devdoped  by  Swift  [43],  but  we  augmented  it  with  an  dght- 
state  Drydmi  wind  modd  used  in  previous  research  [33,  38, 41, 42],  with  some  adjustmoit  fix' 
altitude  and  speed.  The  software  module  used  in  previous  research  to  integrate  the  truth  modd 
proved  to  be  inadequate  for  this  a|q)]icatioa  This  application  involves  an  open-loq[>  aircraft 
(no  flight  ctHttrollo')  which  yidded  a  stiff  diffeimitial  equation  (widdy  sq)arate  dgenvalues) 
that  had  to  be  integrated.  We  ended  up  using  MATRD^  to  int^rate  this  differential  equation 
because  of  this  defidoicy.  The  integrated  values  woe  sampled  at  tbe  measurmnent  sampling 
frequency  and  stored  in  a  data  file,  along  with  the  aircraft  uq)ut  vector,  that  was  used  by  tiie 
MMAE  as  tbe  source  of  its  measurement  and  iiqnit  vectors. 

3.3.3  Diagnostics  and  Performance  Measures 

We  used  several  variables  from  tbe  preceding  equations  as  diagnostic  tools  to  verify 
the  operation  of  the  MMAE.  The  individual  scalar  donents  of  tiie  residual  veefix  ( r )  were 
used  to  verify  the  opoation  of  the  elemental  Kalman  filters.  Tbe  residuals  in  a  well  tuned 
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Kalman  filter  with  the  correct  model  will  be  substantially  smaller,  relative  to  the  filter- 
computed  covariance,  than  the  ones  in  the  incorrect  filters.  Thus,  the  elements  of  the  residual 
vector  gave  a  general  indication  of  how  well  the  Kalman  filters  were  tuned.  The  { *}  term 
and  conditional  probability  density  in  (9)  were  used  to  verify  the  operation  of  the  hypothesis 
testing  algorithm.  The  incorrect  Kalman  filters  should  have  much  larger  values  for  these  terms 
(i.e..  larger  values  of  r).  and  when  a  failure  occurred,  the  { *}  term  gives  an  indication 
of  how  fast  the  conditional  probabilities  will  change.  All  these  terms  aided  in  diagnosing  and 
verifying  the  operation  of  the  MMAE. 

The  best  indicator  of  the  MMAE  performance  was  a  smoothed  version  of  the 
individual  filter  conditional  probability.  PtOi )  in  (10),  that  is  generated  by  the  hypothesis 
testing  algorithm.  Previous  research  [33, 42]  has  shown  that  some  smoothing  of  the  filter 
conditional  probabilities  is  needed  to  prevent  false  declarations.  For  our  implementation,  we 
smoothed  the  probabilities  over  a  variable  length  data  window  (PWINSIZ),  that  was  usually 
set  at  10  samples,  and  then  tested  for  excursions  above  the  threshold  levd.  If  the  probability, 
averaged  over  the  data  window,  exceeded  the  threshold  levd,  a  failure  was  dedared  that 
oHiesponded  to  the  Kalman  filter  modd  that  h«i  the  highest  probability. 

The  conditional  probabilities  fm  eadi  of  the  Kalman  filters  were  plotted  versus  time 
for  various  failure  scenarios  to  visualize  the  MMAE  performance.  A  typicd  plot  is  shown  in 
Hgure  7,  where  a  right  devator  failure  is  simulated  at  2  seconds.  Note  that  the  probability  in 
the  fully  functional  filter  is  high  while  all  the  oth^  filters  are  essentially  zero  until  the  failure 
occurs,  then  the  probability  in  the  right  devatm  failure  filter  grows  as  the  probability  in  the 
fully  functional  filter  decreases.  This  behavior  was  observed  to  occur  fin*  most  of  the  failure 
sceruuios.  so  the  information  was  combined  irrto  me  plot,  shown  in  Figure  8.  This  plot  shows 
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how  the  probabilities  for  each  of  the  filters  grows  when  a  failure  that  matches  its  own  model 
occurs.  We  found  that,  in  all  cases  except  an  aileron  failure,  the  probability  in  the  fully 
functional  filter  was  an  inverse  of  the  probabilities  shown  in  each  plot  (i.e..  one  minus  that 
probability  value)  and  the  probabilities  assigned  to  file  othm*  filters  were  minimal.  For 
instance,  the  right  elevator  failure  shown  in  Figure  7  is  rqiresented  by  the  plot  of  the  right 
elevator  filter  in  Figure  8.  You  can  see  that  the  probability  in  the  right  elevator  failure 
matches  in  both  plots,  while  the  fully  functional  probability  is  simply  the  inverse  of  the  right 
elevator  probability  and  all  the  otho-  probabiliti^  are  at  their  minimums.  The  plotting  format 
in  Figure  8  will  be  used  throughout  the  rest  of  this  document,  mccqit  as  noted.  Also,  to 
prevmit  discontinuities  in  the  text,  we  will  ffoap  full  page  figures,  such  as  Figure  7  and  Hgure 
8.  at  the  end  of  the  section  in  which  they  are  referenced. 
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Chapter  IV:  Results 


4.1  Chapter  Overview 

This  ch{y)tier  preseois  the  results  of  our  investigation  into  the  failure  detection 
capabilities  of  the  MMAE  for  the  LAMBDA  flight  vehicle.  These  results  are  (vganized  to 
correspond  with  the  research  questions  presented  in  Section  1.5,  Research  Questions.  In 
Section  4.2,  Decision  Convergence,  we  present  the  MMAE’s  baseline  decision  convergence 
poformance.  Section  4.3,  Dedsion  Switching,  presents  the  results  of  various  attempts  to 
improve  this  performance  and  the  effect  that  either  an  incorrect  decision  or  improper 
initialization  has  on  the  MMAE’s  perf(»mance.  Finally,  in  Secdm  4.4,  Unmodeled  Effects, 
we  present  the  MMAE’s  performance  at  various  flight  conditions  located  on  the  boundaries  of 
the  flight  envelope.  Throughout  this  dupter  we  have  tried  to  present  a  modest  number  of 
results  to  illustrate  our  findings.  An  exhaustive  set  of  results  is  on  file  with  Dr.  Peter 
Maybeck,  Air  Force  Institute  of  Technology,  School  of  FJigineering,  Wright-Patterson  Air 
Force  Base,  Ohio. 

4.2  Decision  Convergence 

The  baseline  performanoe  of  the  MMAE  developed  in  Chapter  m  is  shown  in 
Figure  9.  In  Section  3.3.3  we  described  die  fonnat  of  this  plot,  where  each  strip  plot  shows 
the  probabilities  of  the  correct  filter  for  a  sqiarate  simulation  run.  For  instance,  tbe  no-fail 
strip  plot  shows  the  probatHlities  assigned  to  the  no-failure  hypothesis  during  a  simuladon  run 
in  which  there  were  no  failures  induced.  Likewise,  the  right  elevator  strip  plot  shows  the 
probabilities  assigned  to  the  right  elevator  failure  l^potfaesis  during  a  simuladrHi  run  in  vdiich 
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a  right  devator  failure  was  induced  at  2  seconds,  and  so  forth  fen*  each  of  the  failure 
hypotheses  that  were  modeled.  Note  that  the  no-failure  hypothesis  results  are  presented  first, 
followed  by  all  the  actuator  failure  hypotheses  results,  and  then  the  sensw  failure  hypotheses 
results.  Also,  each  plot  is  the  average  probabilities  over  a  10-run  Monte  Carlo  simulation. 
Normally  a  ±la  plot  is  superimposed  on  a  multiple  run  simulation  to  indicate  how  variable 
each  run  is  compared  to  the  average  run.  In  most  cases  the  ±la  plot  and  the  10-run  average 
were  so  dose  that  it  was  impossible  to  distinguish  them.  This  shows  that  most  single  run 
performances  are  very  dose  to  the  10-run  average  performance,  which  was  verified  by  plotting 
some  single  run  simulations.  The  plots  that  are  presented  in  this  thesis  are  10-run  Monte 
Carlo  averages  without  the  ±la  plots,  unless  othmwise  noted. 

We  consider  convergence  to  be  when  one  of  the  smoothed  filter  probabilities  remains 
above  the  dedsion  threshold,  while  the  other  probabilities  remain  below  the  dedsion  threshold. 
We  used  a  dedsion  threshold  of  0.3  (30%),  whidi  will  be  shown  as  a  dotted  line  on  the 
various  plots  of  the  MMAE  performance  throughout  this  document  The  baseline  performance 
in  Figure  9  shows  that  the  MMAE  convert  to  the  correct  failure  condition  within  3.2 
seconds  fw  all  failure  conditions. 

In  following  sections  we  experiment  with  various  techniques  to  improve  the  MMAE 
perfrxmance.  To  identify  these  techniques  dearly,  the  vdues  of  the  parameters  tiiat  are 
adjusted  for  these  various  techniques  are  listed  in  the  ctqytion  of  the  figure.  We  will  test  the 
MMAE  perfmmance  using  three  different  Kalman  filter  bank  designs,  which  will  be  identified 
by  the  Design  #  in  the  figure  c^on.  Then  we  will  experiment  witit  different  values  of  the 
{ *}  tmn  in  (9)  on  Page  23.  That  value  will  be  identified  as  the  "Dot”  term  in  the  figure 
cqtticHL  We  also  experimmit  with  diffoent  size  data  windows  over  which  the  probabilities  are 
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smoothed.  The  window  size  will  be  indicated  by  "PWINSIZ"  (probability  window  size)  in  the 
caption  of  Figure  9.  Lasdy,  we  will  try  propagating  the  Kalman  filter  estimates  a  few  sample 
periods  without  updating.  The  number  of  sanqile  pmiods  that  the  estimates  are  propagated  is 
indicated  by  "NPROF'  (number  of  propagations)  in  the  caption  of  Figure  9. 

Some  experimentation  with  various  commanded  iiqiuts  was  required  to  attain  this 
baseline  performance.  We  found  that  the  commanded  iiq>ut  had  to  properly  excite  all  the 
aircraft  modes  for  the  MMAE  to  converge  wdl.  For  example,  a  5  Hertz  dither  ii^ut  in  all 
channels  does  not  excite  any  of  the  aircraft  modes  and  Hgure  10  shows  that  the  MMAE 
cannot  converge  to  the  proper  decision.  Note  that  the  probal^ties  in  the  filtn  with  the  wrong 
hypothesis  are  not  at  their  minimums  (0.001).  Since  the  probabilities  are  forced  to  sum  to  1, 
thdr  combined  effect  is  to  lower  the  probability  of  the  filter  with  the  correct  hypothesis  so  that 
it  is  below  or  at  the  decision  threshold,  thus  the  MMAE  does  not  convnge.  This  occurs  whmi 
the  Kalman  filter  residuals  are  all  about  equal  in  size  (relative  to  the  filter'ccxnputed  residual 
standard  deviations),  so  all  of  the  filter  hypotheses  look  equally  likely,  and  die  MMAE  must 
allow  some  probability  fm  the  wrong  hypotheses.  To  prevent  this  bdiavicH’,  the  commanded 
iiqnit  had  to  have  the  cmiect  frequency,  strength,  and  lag  betwemi  right  and  left  actuators. 

To  find  the  frequency  of  an  irqnit  that  would  provide  good  failure  identification 
perftxmance,  we  started  with  the  eigenvalues  of  the  continuous-time  model  and  then 
mqperimented  with  various  combinations  of  the  natural  fiequencies.  Fm  the  design  point  in  the 
flight  mivelope  these  dgenvahies  were  -2.928  ±  j  6.68  and  -1.516  ±  j  0.260  rad/sec  in  the 
Irmgimdinal  axis,  and  0,  -6.724,  -0.9424  ±  j  3.625  rad/sec  in  die  longitudinal  axis. 

Esqiierimenting  with  iiqnits  at  the  natural  frequencies  of  6.68  and  3.625  rad/sec,  we 
found  that  the  magnitude  of  the  iiqnits  had  to  exceed  a  certain  value  to  produce  good  MMAE 
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performance.  If  the  iiq)ut  was  bdow  this  threshold,  the  aircraft  modes  were  not  excited 
enough  and  the  performance  was  similar  to  Hgure  10.  Once  this  threshold  was  exceeded,  the 
MMAE  performed  quite  well.  Also,  inaeasing  the  magnitude  above  this  threshold  did  not 
enhance  the  MMAE  perfomance.  We  found  that  a  ±10  degree  roll  axis  iiq)ut  at  3.625  rad/sec, 
±15  degree  pitch  axis  iiq)ut  at  6.678  rad/sec,  and  a  ±20  degree  yaw  axis  input  at  6.678  rad/sec 
produced  good  results.  While  these  inputs  seem  rather  large,  the  aircraft  response  was  actually 
quite  small  because  the  iiq)uts  were  almost  exactly  mit  of  phase  with  each  othm*,  due  to 
reasons  that  are  e^qtlained  in  the  next  paragriq)h. 

Not  only  does  the  input  need  to  exdte  the  aircraft  modes,  but  a  differential  iiqmt 
(Figure  1 1)  had  to  be  applied  for  the  MMAE  to  distinguish  between  a  right  and  left  actuator 
failure.  A  differential  itqmt  commands  one  actuator  with  a  positive  command  and  the  other 
actuator  with  a  negative  command,  with  positive  and  negative  defined  by  the  sign  of  the 
aircraft  response  to  that  command.  For  example,  if  the  left  elevator  received  a  positive 
conunand.  it  would  turn  upwards  in  an  attempt  to  produce  a  positive  pitch,  and  if  the  right 
etevanv  received  a  negative  command,  it  would  turn  downwards  in  an  attmnpt  to  produce  a 
negative  pitch.  Note  that  die  aileron’s  response  to  the  differential  input  differs  fix>m  the  rudder 
and  efevatOT  responses.  If  the  left  aileron  received  a  positive  command,  it  would  turn 
downwards  in  an  attmnpt  to  produce  a  positive  roll,  and  if  the  right  aileron  received  a  negative 
command,  it  too  would  turn  downwards  in  an  attempt  to  produce  a  negative  roll 

If  both  commands  are  equal,  and  both  actuators  are  wt^dng  prqierly,  then  the  net 
response  is  zero  according  to  our  model  ftv  the  LAMBDA.  Good  engineering  judgment  teds 
us  that,  if  the  elevators  are  used  as  described  above,  there  would  be  some  rolling  motion 
induced  by  the  devatws.  However,  the  LAMBDA  model  does  n(H  predict  this  response  due  to 
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the  lack  of  cross-axis  terms  that  was  noted  in  Section  1.5.  We  have  assumed  that  this  cross¬ 
coupling  between  the  axes  is  small  due  to  the  small  mommit  arm  of  the  aoss-axis  flight 
surface  (i.e.,  the  elevator  roll  mmnent  arm  is  small  compared  to  the  ailm'on  roll  moment  arm). 

When  a  non-differential  iiq)ut  is  applied  (i.e.,  when  the  right  and  Idt  actuator 
commands  are  the  same),  th^  is  insuffident  information  to  detomine  the  difference  between 
a  right  or  left  actuator  failure.  We  found  that  the  MMAE  divides  the  probability  equally 
between  both  hypotheses,  when  dther  actuator  fails.  To  identify  right  vs  left,  we  used  a 
differential  iiqtut  so  that  if  one  actuator  fails,  the  net  response  is  no  longer  zero,  and  the  failure 
becomes  quite  evident.  We  also  found  that  the  differential  inputs  had  to  be  slightly  shifted 
relative  to  each  other  (about  0.05  wavdmigths)  to  provide  some  ouq)ut  so  that  the  sensors 
were  exdted  and  a  sensor  failure  could  be  detected.  After  expoimenting  with  a  few  different 
iiq)uts,  we  found  that  the  iiqmt  shown  in  Hguie  1 1  provided  good  failure  detection 
perfonnance.  This  plot  shows  the  commanded  iiq>ut  to  each  actuator  starting  with  the  right 
and  left  devaiors  (rt  dev  and  It  dev),  the  right  outboard,  right  inboard,  left  inboard,  left 
outboard  flaps  (rof,  rif,  lif,  and  lof),  the  right  and  left  ailerons  (rt  ail  and  It  ail),  and  finally,  tlte 
right  and  left  rudders  (rt  rdr  and  It  rdr). 

The  MMAE  seemed  to  be  quite  sensitive  to  the  magnitude  and  ftequency  of  the  iiqxits. 
Hgure  10  shows  the  performance  of  the  MMAE  when  the  ftequency  was  not  at  the  natural 
fiequendes,  and  similar  results  were  obtained  wlten  the  magnitude  was  too  low.  Also,  if  the 
iiqmt  was  not  shifted  enough  to  exdte  the  sensors,  the  perfonnance  would  d^rade  nqiidly. 
This  seems  to  indicate  that  the  iiqmt  has  to  be  rather  predse  in  its  magnitude,  ftequency,  and 
phase  relationship,  to  provide  good  failuie  idenfificafion.  This  would  indicate  that  normal 
oHninanded  iiqmts  fimn  a  pilot  will  not  provide  good  MMAE  perfonnaiice,  so  a  purposeful 
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dither  must  be  implied.  This  dither  must  exdte  cdl  the  failure  modes  that  are  modeled  in  the 
MMAE  so  the  MMAE  can  converge  to  tte  correct  failure  condition. 

The  ii^ut  that  was  used  for  this  thesis  is  rather  large,  but  because  the  iiq>uts  are  almost 
exactly  out  of  phase  with  each  other,  the  aircraft  response  is  acmally  quite  small,  as  shown  in 
Figure  12.  This  shows  that  the  forward  velocity  (u)  changes  only  3  ft/sec,  which  is  quite  small 
compared  to  design  point  velocity  of  180  ft/sec.  Also,  the  angle  of  attadc  (alpha)  varies  only 
±  O-S**,  the  pitch  rate  (q)  varies  ±S7sec,  and  the  pitch  angle  varies  about  ±P.  In  the  lateral 
axis  the  sideslip  angle  (beta)  varies  only  i0.8°,  the  roll  rate  (p)  varies  ±2.S7sec,  the  roll  angle 
(phi)  varies  ±0.8®,  the  yaw  rate  (r)  varies  ±4®/sec,  and  the  yaw  angle  (psi)  varies  ±1°.  The 
design  of  an  optimal  iiq>ut  is  beyond  the  scope  of  this  thesis,  so  othm*  iiq>uts  were  not  tried 
once  a  good  iiq)ut  was  found. 

Figure  9  shows  that  the  MMAE  does  converge  to  tte  prc^  failure  detection  decision 
in  less  than  1  second  for  most  of  the  failure  conditions.  However,  a  right  aileron  failure 
produces  some  confusion  with  the  sideslip  and  yaw  rate  sensms  fm  about  1  second  before  it 
finally  convo-ges  to  the  correct  failure  decision,  as  shown  in  Figure  13.  Note  that  Hgure  13  is 
a  plot  of  a  single  10-run  Monte  Carlo  simulation  whoe  the  right  aileron  actuatm  is  failed  at  2 
seconds;  as  t^tposed  to  results  such  as  Figure  9,  in  which  each  time  history  is  derived  from  a 
sq)arate  10-run  Monte  Carlo  simulation.  Similarly,  a  left  aileron  failure  produced  some 
confusion  for  almost  2  seconds,  as  shown  in  Figure  14.  The  difference  between  these  results 
might  possibly  be  due  to  the  phasing  of  their  respective  iiqnits. 

None  of  the  other  failure  conditions  produced  the  confusion  observed  fm  an  aileron 
failure,  which  cmnpdled  us  to  investigate  the  cause  fm  this  behavior.  The  noise 
measurements  that  were  used  to  determine  the  values  for  in  (3)  on  Page  23.  showed  ftiat  die 
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sideslip  and  yaw  rate  sensors  woe  the  least  noisy  of  all  the  sensors.  The  standard  deviations 
for  the  residuals  that  are  formed  using  these  sensor  measurements  would  be  small  compared  to 
the  other  sensors.  Therefore,  any  residuals  that  are  larger  than  expected  from  these  smisors 
would  show  up  very  quickly,  and  this  would  be  interpreted  as  a  failure  of  one  of  these  sensors. 
Eventually,  the  residuals  from  the  other  sensors  would  become  large  enough  to  correct  any 
erroneous  declarations,  but  the  end  result  is  that  the  MMAE  is  more  likely  to  declare  a  failure 
in  these  senstvs,  if  there  are  residuals  that  are  largm*  than  expected.  In  Section  4.3.1  we 
present  an  attempt  to  decrease  the  amount  of  time  chat  an  aileron  failure  causes  confusion. 

Figure  9  also  shows  that  a  rudder  failure  condition  took  substantially  longer  to 
converge  to  the  proper  decision  than  any  of  the  other  failure  conditions.  Note  that  the  rudder 
failures  took  about  3  seconds  to  cross  the  50%  probability  threshold  that  appears  as  the  dashed 
line  in  the  plots,  while  most  of  the  other  failures  took  less  than  l  second.  In  Sections  4.3.2, 
4.3.3,  and  4.3.4,  we  present  various  attempts  to  decrease  the  amount  of  time  it  took  for  the 
MMAE  to  converge  fm  a  rudder  failure. 
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4.3  Decision  Switching 


In  this  section  we  present  the  results  of  various  attempts  to  decrease  the  amount  of 
time  that  the  MMAE  took  to  switch  to  the  correct  failure  condition  when  a  failure  occurred. 
We  start  by  tuning  the  Kalman  filters  to  minimize  the  confusion  caused  by  an  aileron  failure. 
We  then  increase  the  mcponential  penalty  for  a  larger  than  expected  residual.  Tbe  MMAE 
design  in  Ch^ter  HI  smcxithed  the  probabilities  to  minimize  false  alarm  declarations,  which 
introduces  a  delay  in  the  decision  convergence.  This  smoothing  is  decreased  in  an  attempt  to 
reduce  the  time  it  takes  for  the  MMAE  to  convage  to  the  correct  failure,  condition.  Lastly,  die 
residuals  from  the  Kalman  filters  are  enlarged  by  propagating  the  state  estimates  a  few  sample 
periods  without  updating.  The  residuals  in  the  Kalman  filtos  with  the  wrong  hypotheses 
become  even  larger,  which  causes  the  MMAE  to  converge  even  faster. 

4.3.1  Kahnan  Filter  Tuning 

We  expoimented  widi  increasing  the  measurmnmit  variances  in  the  sideslip  and  yaw 
rate  soiscms,  in  an  attempt  to  decrease  the  confusion  (ambiguous  and/cM-  false  declarations  of 
the  failure  of  these  two  sensors)  during  an  ailercm  failure  that  was  noted  in  the  previous 
secdotL  We  developed  three  filter  designs;  design  one  used  the  senscx’  noise  variances  that 
woe  estimated  from  sensor  measurement  data,  design  two  used  nine  times  the  sensor  noise 
strength  estimates  (i.e..  increasing  the  noise  standard  deviations  by  a  factor  of  three),  and 
design  three  used  25  times  those  estimates  (multiplying  the  standard  deviations  by  a  factcv  of 
five).  The  MMAE  perfonnanoe  using  design  one  was  presented  in  the  previous  section  as  the 
baseline  performance  (Hgure  9),  alcmg  with  the  right  aileron  performance  (Figure  13)  and  the 
left  aileron  performance  CFigure  14).  Ihe  MMAE  performance  using  design  two  is  shown  in 


47 


Figure  IS.  with  the  right  aileron  failure  performance  shown  in  Figure  16  and  the  left  aileron 
failure  performance  shown  in  Hgure  17.  Likewise,  the  MMAE  p^cmnance  using  design 
three  is  shown  in  Hgure  18.  with  the  right  ailo'on  failure  performance  shown  in  Figure  19  and 
the  left  aileron  failure  performance  shown  in  Figure  20. 

The  best  performance  was  obtained  using  design  two.  Figure  15  shows  that  the 
convergence  time  for  an  aileron  failure  using  design  two  was  half  of  that  fm  design  one 
(Bgure  9).  Figures  16  and  17  show  that  this  improvement  in  convergence  time  is  primarily 
due  to  a  significant  decrease  in  the  amount  of  confusion  caused  by  an  aileron  failure. 
Conqiaring  the  results  from  design  one  (Figure  13  and  Figure  14)  with  those  from  design  two 
(Hgure  16  and  Hgure  17).  we  see  that  design  two  has  eliminated  the  second  hun^  in  the 
probabilities  fen*  a  sideslip  and  yaw  rate  sensor  failure.  This  caused  the  MMAE  to  converge 
much  faster  to  tlm  anrect  failure  conditioiL 

Figures  18,  19,  and  20  show  that  the  perfcarmance  for  design  three  is  almost  exactly 
the  same  as  the  performance  fm  design  two,  but  the  probabilities  showed  an  increase  in 
fluctuations.  We  noted  before  that  these  results  are  an  average  of  10  Monte  Carlo  simulations, 
therefore  fluctuations  in  these  plots  would  indicate  even  more  severe  fluctuations  during  flight 
testing,  which  would  result  in  a  larger  numbo’  of  false  alarms,  which  are  deemed  unaccq>table. 
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Figure  15.  Destgn  #2,  Dot  =  -0.5,  PWINSE  =  10,  NPROP  =  1 
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4.3.2  Exponential  Penalty  Increase 

Previously  we  noted  that  the  rudder  failure  identification  required  a  substantially  longer 
time  to  converge  than  all  otho*  failure  conditions.  To  decrease  this  convergence  tune,  we 
noted  that  the  -  V6  in  the  { *}  term  in  the  equation  fin-  the  conditional  density  fimction, 

6  =  1 

can  be  viewed  as  a  penalty  for  having  a  larger  than  e3q)ected  lesiduaL  By  increasing  this  term, 
the  same  residual  will  produce  higher  penalties  and  cause  the  MMAE  to  change  its 
probabilities  faster  when  a  failure  occurs,  thus  acting  as  a  decision  convergraice  gain. 

We  experimented  with  values  of  -1  and  -2  to  rq)lace  in  the  "Dot"  term  of  (9). 
Using  this  strategy  with  Design  #1,  we  found  that  the  convergence  time  (when  the  probability 
was  above  the  30%  probability  threshold)  for  a  rudder  failure  decreased  fiom  3.2  seconds  for  a 
"Dot"  term  of  -Vi  (shown  in  Hgure  9),  to  1.8  seconds  for  a  "Dot"  term  of  -1  (shown  in  Hgure 
21),  and  finally  to  0.8  seconds  fw  a  "Dot"  term  of  -2  (shown  in  Figure  22).  We  found  that 
the  fastest  convergence  times  were  obtained  with  Design  #2  and  a  "Dot"  tenn  of  -1,  as  shown 
in  Figure  23.  However,  this  increase  in  the  "Dot"  term  also  produced  much  larger  fluctuations 
in  the  probabilities.  The  perfixmance  for  a  single  Monte  Carlo  run  is  shown  in  Hgure  24, 
whidi  indicates  diat  false  alarms  would  qKX’adically  occur  during  actual  flight  testing,  which  is 
considered  to  be  unaccq)table  performance.  To  avtnd  these  false  alarms,  we  used  Design  #1 
with  a  "Dot"  term  of  -1  (Hgure  21).  Note  diat,  by  using  Design  #1,  tte  aileron  failures  take 
up  to  one  second  longer  than  they  do  fm  Design  #2,  but  the  advantage  is  the  lack  of  false 
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alarms.  For  this  design  tbe  convergence  to  the  cmrect  failure  condition  occurs  in  less  than  2 
seconds  ftnr  all  the  failure  conditions,  without  any  false  alarms  once  convergence  is  attained. 
Other  values  of  the  "Dot”  tom  and  the  sensor  lUMse  strengths  would  probably  improve  the 
performance  even  more,  but  time  constraints  prevmed  further  experimentation. 
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Figure  21.  Design  #1.  Dot »  -1,  PWINSIZ  »  10,  NFROP  »  l 
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Figure  22.  Design  #1,  Dot  =  -2,  FWINSIZ  =  10.  NFROP  =  1 
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4.3.3  Decreased  Probability  Smoothing 


We  experimented  with  decreasing  the  size  of  the  data  window  over  which  the 
probabilities  are  smoothed  (PWINSIZ)  as  another  possible  means  of  decreasing  the 
convergence  time.  Figure  23  shows  the  results  of  decreasing  the  window  size  fiom  10  to  3. 
Note  that  the  performance,  shown  in  Hgure  23,  does  not  improve  significantly  over  the 
performance  shown  in  Hgure  21,  while  the  fluctuations  in  the  probabilities  increase,  which 
indicates  an  increase  in  false  alarms  would  occur  during  flight  testing.  Decreasing  the  window 
size  increases  the  false  alarms  because  high  frequency  fluctuations  in  the  probabilities  are  no 
longer  smoothed  over  several  time  periods.  SiiKe  decreasing  the  smoothing  does  not  inq)rove 
the  performance  significantly,  but  it  does  increa^  the  number  of  false  alarms,  we  retained  the 
original  data  window  size  (PWINSIZ  =  10). 
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Figure  25.  Destgn  #1.  Dot  =  -1.  PWINSIZ  =  5.  NFROP  =  1 


4.3.4  Enlarged  Residuals 

Another  method  of  possibly  decreasing  the  decision  convergence  time  would  be 
to  propagate  the  Kalman  filter  state  estimates,  without  updating,  for  a  few  sample 
times  while  still  monitoring  the  residuals.  This  allows  the  residuals  in  all  the  filters  to 
grow  much  larger  since  the  usual  measurement  updates  are  no  long^  correcting  the 
state  estimates  toward  the  actual  measurements,  thereby  masking  the  impact  of  an 
incorrect  hypothesis.  The  results  for  propagating  two  sample  periods  (NPROP  =  2) 
before  updating  again  are  shown  in  Figure  26.  These  results  show  no  significant 
improvement  in  the  MMAE  performance  using  this  strategy,  and  a  slight  increase  in 
the  fluctuations  of  the  probabilities. 

Figure  27  shows  the  results  of  propagating  5  sample  periods  before  updating. 
These  results  are  quite  poor,  which  could  be  due  to  a  poor  input  for  identifying  these 
failures.  This  method  essentially  changes  the  measurement  sampling  time  because  the 
measurements  are  used  only  when  the  estimates  are  updated,  which  is  every  NPROP 
sample  periods.  A  new  sampling  time  has  a  profound  impact  on  the  system  modes, 
thus  a  new  input  might  be  needed  to  excite  the  failure  modes. 
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Figure  26.  Desiga  #1.  Dot «  -1,  FWINSIZ  »  10.  NFROP  «  2 
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4.3.5  Initialization  Effects 


The  probabilities  in  the  MMAE  need  to  be  initialized  because  they  are  used  to 
calculate  the  current  hypothesis  probability.  Of  all  the  failure  hypotheses,  the  fully  functional 
hypothesis  is  most  likely  to  be  the  correct  initial  hypothesis  for  a  well  maintained  aircraft. 
Therefore,  the  MMAE  should  be  initialized  such  that  the  Kalman  filter  with  the  fully 
functional  hypothesis  has  very  high  probalMty  and  the  other  Kalman  filters  equally  divide  the 
remaining  probability.  In  previous  simulations,  we  initialized  the  MMAE  such  that  the  fully 
functional  hypothesis  Kalman  filter  has  a  prd)ability  of  0.75  and  the  other  Kalman  filters  have 
a  probability  of  0.0179,  so  that  the  total  probability  sums  to  one. 

To  investigate  the  performance  of  an  MMAE  that  has  either  been  initialized  poorly  or 
has  momentarily  converged  on  the  wrong  hyptxhesis,  several  simulations  were  performed  with 
the  MMAE  intentionally  initialized  incorrectly.  F(»'  each  of  the  simulations,  one  of  the 
Kalman  filters  was  initialized  with  a  probability  of  0.95  and  the  others  with  a  probability  of 
0.0036.  Several  simulations,  one  for  each  failure  condition,  were  run  to  observe  the  amount  of 
time  it  took  for  the  MMAE  to  cmiverge  to  the  cmrect  failure  condition.  Figure  28  shows  the 
perf(»mance  of  the  MMAE  when  each  filto*  is  initialized  inccMTectly  and  a  no  failure 
simulation  is  performed.  Note  that  this  plot  diffos  significantly  fipom  previous  plots.  In 
Figure  28  the  plots  show  only  the  probat^ties  fot  the  no-fail  filter  (whidi  is  the  cmrect  failure 
hypothesis).  The  filters  that  are  listed  along  the  y-axis  are  the  filters  that  are  initialized  to 
0.95.  Fot  instance,  the  right  elevator  plot  shows  tte  no-faihiie  probability  when  the  right 
elevatOT  filter  hypothesis  is  initialized  to  0.95.  Tbe  MMAE  converges  to  the  no-failure 
hypothesis  usually  within  one  second,  except  fOT  the  rudder  failure  hypotheses  where  it  taifps 
about  two  seconds.  Tbe  same  behavior  was  found  for  other  failure  simulations. 
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4.4  Umnodeled  Effects 

We  investigated  the  failure  detection  performance  of  the  MMAE  against  unmodded 
effects  by  simulating  its  performance  at  the  comers  of  the  flight  envdope  shown  in  Figure  29, 
with  the  MMAE  design  using  the  nominal  conditions  (shown  by  the  circled  dot  in  Figure  29). 


At  point  A  (high  vdodty,  high  weight)  and  point  B  (high  vdocity,  low  wdght),  we  found  that 
the  MMAE  perfonnance  (Hgure  30)  was  almost  exactly  the  same.  Hgure  30  shows  that  the 
MMAE  still  converges  to  the  correct  failure  coixlition  in  less  than  i.s  seconds  fw  all  failures 
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conditions,  wMch  is  better  than  at  the  design  point  (2  seconds).  However,  there  are 
significantly  high^  fluctuations  in  the  probabilities,  which  indicate  that  quite  a  few  momentary 
false  alarms  will  occur  during  acmal  flight  testing.  This  is  particularly  true  for  the  aileron- 
failure  Kalman  filters,  which  show  even  higher  fluctuations  than  any  of  the  other  filters.  This 
is  commensurate  with  our  earlier  observations  that  these  filters  have  the  most  trouble 
converging  to  the  correct  hypothesis. 

We  also  looked  at  the  MMAE  performance  at  the  low  end  (with  respect  to  velocity 
and  dynamic  pressure)  of  the  flight  envelope.  Hguie  31  shows  extimndy  poor  MMAE 
performance  at  point  C,  and  comparable  performance  was  found  for  point  D.  The  performance 
in  Figure  31  is  totally  unacceptable;  therefore,  some  modification  of  the  MMAE,  such  as  gain 
sdieduling,  is  required  to  provide  the  desired  perfmmance  at  these  points.  Such  gain 
scheduling  would  not  be  difficult  to  implement  and  it  is  highly  recoimnended  fm*  a  final 
system  configuratioiL 
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Figure  30.  MMAE  PeifOTmance  at  Flight  Condition  Pcnnt  A. 
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Chapter  V:  Conclusions 


5.1  Chapter  Overview 

In  this  chapter  we  present  the  conclusions  that  can  be  drawn  from  our  findings  that 
were  presmited  in  Chapter  IV.  Tb^*  conclusions  are  mganized  to  correspond  with  the  research 
questions  that  were  presented  in  Section  l.S.  In  Section  S.2  we  look  at  the  al^ty  of  the 
MMAE  to  converge  to  the  correct  failure  condition.  In  Section  5.3  we  evaluate  the  various 
methods  that  were  tried  to  improve  the  rate  of  convergence  to  the  correct  failure  conditioa  In 
Section  5.4  we  draw  conclusions  about  some  aspects  of  the  robustness  of  the  MMAE  based  on 
its  performance  at  the  boundaries  of  the  flight  envdope.  Finally,  we  dose  with  our 
recommendations  for  future  research  in  Section  5.4. 

5.2  Decision  Convergence 

We  found  that  the  MMAE  did  converge  to  the  (x»iect  failure  condition  if  a  command 
iiq)ut  (dther  a  puiposefiil  command  or  a  dither  to  enhance  detection)  is  provided  that  exdtes 
all  the  failure  modes  ctHrectly.  We  found  such  a  dither  by  using  the  dgenvalues  of  the 
continuous-time  plant  modd  and  then  expoimenting  until  good  results  were  achieved.  This 
ditha:  had  diffo’eat  characteristics  (magnitude  and  frequency)  in  each  axis,  and  had  to  be  a 
difidential  input  (when  right  actuator  is  positive,  the  left  actuator  is  n^ative)  that  was  slightly 
shifted  in  phase  to  provide  enough  exdtement  for  the  MMAE  to  detect  a  sensm  failure.  We 
also  found  that  diere  were  thr  in  magnitude,  frequency,  and  differential  phase  that  had 
to  be  exceeded  for  the  MMAE  to  converge.  Once  the  thresholds  were  crossed,  additional 
magnitude  or  phase  did  not  oihance  convergence.  The  frequency  of  the  dither  fx'obably  has  to 
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be  within  a  certain  bandwidth  cratered  arouiul  the  natural  frequencies.  This  points  to  the 
possibility  of  using  spectral  analysis  to  design  optimal  inputs. 

5.3  Decision  Switching 

We  found  that  the  decision  convergence  rates  differed  dramatically  dq)ending  on  the 
failure  condition,  and  various  methods  did  improve  the  convergence  rate,  particularly  for  the 
failure  conditions  that  suffered  from  slow  detection  rates.  All  of  these  methods  involved 
trading  an  increase  in  false  alarms  for  an  increase  in  convergence  performance.  The  methods 
that  we  tried  were  tuning  of  the  Kalman  filters,  increasing  the  exponential  penalty  for  larger 
than  expected  residuals,  decreasing  the  data  smoothing,  and  propagating  without  tqxlating  the 
state  estimates  to  enlarge  the  residuals,  particularly  the  ones  associated  with  an  incorrect 
hypothesis. 

We  made  a  cursory  examination  of  the  eigenvalues  of  the  aircraft  dynamics  matrix,  F 
in  (1 1).  and  the  Kalman  filter-computed  residual  covariance  matrix,  in  (8),  to  determine 
why  our  detection  rates  were  loiter  than  those  of  previous  studies  [28,  29,  33,  34,  35,  38, 41, 
42].  We  found  that  the  Kalman  filters  had  to  be  propagated  up  to  100  time  samples  before 
steady  state  was  reached,  when  computing  the  steady  state  Kalman  filter  gains.  The  aircraft 
eigenvalues  were  widely  sqiarated  because  we  were  using  a  model  that  included  all  the  aircraft 
modes.  This  initially  caused  some  proUems  with  the  software  integration  package,  but  by 
using  a  lower  order  integration  routine  and  subintervals  for  integratimi,  we  drcumvented  this 
problem.  An  aircraft  model  based  on  a  abort  period  aj^voximation  might  have  prevented  this 
problem,  and  it  would  have  reduced  the  numb»'  of  aircraft  states,  along  with  possibly 
improving  the  convergence  rate.  It  would  seon  that  the  difference  in  convergmioe  rates  is 
most  likdy  due  to  the  open-loop  model  of  the  aircraft  that  was  used  fix'  this  research. 
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We  tried  tuning  the  Kalman  filter  models  to  decrease  the  ambiguities  that  were 
observed  when  an  aileron  failure  was  modeled.  This  method  was  quite  effective  in  decreasing 
the  confusion,  as  presented  in  Section  4.3.1.  We  found  that  an  aileron  failure  was  identified  in 
about  half  the  time  it  took  to  converge  prior  to  the  filters  bdng  tuned.  Some  experimentation 
was  required  to  attain  this  performance.  If  the  values  in  the  measurement  noise  covariance 
matrix  were  increased  too  far,  fluctuations  in  the  probabilities  occurred  and  the  false  alarm  rate 
would  increase.  The  fluctuations  caused  the  correct  hypothesis  filter  probabilities  to  dip  below 
the  threshold,  but  the  other  filters  rarely  jumped  above  the  threshold.  This  behavior  either 
caused  a  misidentification  of  "no  failures,”  or  sin:q>ly  confusion  since  none  of  the  hypotheses 
probabilities  were  above  the  threshold.  We  could  compensate  for  this  increase  in  false  alarms 
by  lowering  the  decision  threshold,  as  long  as  the  fluctuations  were  not  too  severe.  Also,  this 
method  worked  well  only  up  to  a  cotain  point,  after  which  the  performance  did  not  improve. 
The  precomputed  values  for  the  measurement  noise  variances  were  used  by  the  Kalman  filters 
to  determine  how  large  the  residuals  should  be,  so  larger  values  fm  the  measurement  noise 
variances  would  cause  the  filters  to  expect  larger  residuals,  wMch  would  mansk  the  sensm 
measurment  residuals  and  make  it  more  difficult  to  detect  a  failure. 

We  found  that  a  rudder  failiue  took  substantially  longer  to  converge  than  all  other 
failure  conditions.  This  could  be  due  to  the  shorter  moment  arm  for  the  rudders  when 
conqiared  to  other  flight  control  surfaces.  A  ruddo*  iiq>ut  will  take  longer  to  produce  a 
substantial  ouqnit  due  to  this  shorter  moment  arm,  thus  the  Kalman  filter  residuals  will  take 
longer  to  build  and  the  MMAE  will  take  long^  to  detect  this  failure.  Some  improvement  was 
attained  by  increasing  the  "Dot"  term  in  (9),  as  shown  in  Section  4.3.2.  Increasing  the  "Dot" 
term  had  the  effect  of  turning  up  the  gain  on  the  decision  convergence,  since  it  not  only 
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caused  the  MMAE  to  converge  faster,  but  it  increased  the  "noise"  (fluctuations)  in  the 
piobalnlities  which  forced  us  to  set  the  probability  decision  threshold  to  50%  to  avoid  false 
alarms  and  ambiguous  failure  detections.  This  method  works  well  as  long  as  the  false  alarm 
rate  is  tolmble. 

We  tried  to  decrease  the  convergence  time  even  further  by  decreasing  the  data 
smoothing.  The  probabilities  were  smoothed  over  10  time  periods,  which  introduced  a  delay 
in  the  dedsion  convergence.  We  found  that  this  delay  was  minimal  since  it  would  be  at  most 
10  sanq>le  periods  (about  1/6  th  of  a  second),  which  is  quite  small  compared  to  the  dedsion 
convergence  times  on  the  order  of  one  second.  As  this  smoothing  was  decreased,  the  false 
alarm  rate  increased,  so  we  kept  the  smoothing  set  at  10  sample  periods. 

Lastly,  we  tried  propagating  the  Kalman  filter  state  estimates  a  few  sample  periods 
without  mei.'siu:mnent  updating,  which  would  cause  the  residuals  to  grow  larger  than  when  the 
updates  do  occur,  which  would  create  even  larger  residuals  in  filters  based  on  incorrect 
hypotheses  about  the  failure  status.  This  technique  did  not  improve  the  perimmance  of  the 
MMAE  very  much  before  the  false  alarm  rate  grew  too  large.  This  method  essentially  changes 
the  measurement  sampling  rate,  which  in  turn  changes  the  syston  dynamics  as  seen  over  a 
single  measurement  sanqtle  period,  which  then  requires  a  new  input  to  exdte  the  failure  modes 
qptitnally.  This  method  might  have  worked  quite  well  if  a  new  irqmt  could  have  been 
developed,  but  time  did  not  aUow  a  full  explmation  of  this  approach. 

5.4  Unmodeled  Effects 

We  found  that  the  MMAE  was  robust  enough  to  provide  fairly  good  performance  at 
the  upper  boundaries  of  the  flight  oivelope,  but  the  performance  at  the  lower  boundaries  was 
inadetpiate.  The  robustness  of  the  MMAE  would  be  improved  by  discretizing  the  flight 
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envel(^,  creatiiig  MMAE  designs  at  those  points,  and  gain  scheduling  the  MMAE  for  the 
current  flight  condition.  The  MMAE  blended  state  estimate  would  provide  the  best  estimate  of 
the  current  flight  condition  and,  by  using  this  estimate  with  gain  scheduling,  feedback  is 
accomplished  which  would  increase  the  robustness  of  the  MMAE.  A  similar  technique  would 
involve  accomplishing  a  sensitivity  study  to  find  the  parameters  to  which  the  MMAE  gains  are 
most  sensitive,  and  gain  scheduling  using  those  parametm.  We  found  that  the  LAMBDA 
model  was  most  sensitive  to  a  change  in  velodty,  but  we  were  not  able  to  determine  the  effect 
of  these  changes  on  the  MMAE. 

5.5  Recommendations  for  Further  Investigation 

There  are  many  areas  that  require  fiirthra'  investigatioa  The  most  significant 
deficiency  in  this  MMAE  is  the  lack  of  cross  axis  coupling  terms  in  the  LAMBDA  model.  If 
these  terms  were  defined  in  future  research,  then  a  mcxe  reasonable  command  input  could  be 
designed  since  the  effects  of  a  failure  in  one  axis  could  also  be  found  in  the  other  axis. 

Another  area  that  would  be  quite  promising  is  the  use  of  reduced  order  models.  If  a  modd 
was  devdoped  using  a  short  period  approximation  for  the  LAMBDA,  then  a  MMAE  design 
using  this  modd  would  reduce  the  computational  loading  and  might  have  even  faster 
convergoice  rates.  Various  gain  scheduling  techniques  need  to  be  considered  to  enhance  the 
robustness  of  the  MMAE  across  the  whde  flight  envdope.  If  detection  of  multiple  failures  on 
the  LAMBDA  is  desired,  then  a  hierarchical  structure  of  filter  designs  needs  to  be  devdoped, 
as  in  previous  research  into  multiple  failure  detection  [29,  33,  34.  3S,  41, 42].  The  MMAE 
c^abilities  of  detecting  partial  failures  on  the  LAMBDA  needs  to  studied  as  in  previous 
research  for  otha*  aircraft  [29,  33,  34,  33, 41, 42].  A  much  more  general  study,  which  would 
be  applicable  to  all  MMAE  designs,  would  be  K)  determine  the  rdationship,  if  any.  between 
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the  eigenvalues  of  the  plant  and/or  the  filters  and  the  convergence  rates.  If  a  relationship 
exists  and  is  predictable,  then  the  MMAE  performance  could  be  determined  prior  to  the  actual 
design  and  simulation.  Lastly,  a  study  on  the  design  of  q)timal  inputs  for  enhancing  failure 
identification  and/or  detection  is  sorely  needed,  particularly  if  a  different  optimal  input  is 
required  at  various  points  of  the  flight  envelc^. 
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